{
 "cells": [
  {
   "cell_type": "markdown",
   "source": [
    "## 5.2 基本功能 Essential Functionality"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "source": [
    "# 模块导入\r\n",
    "import os, sys\r\n",
    "sys.path.append(os.path.dirname(os.getcwd()))\r\n",
    "import numpy\r\n",
    "import pandas\r\n",
    "from dependency import arr_info"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "### 5.2.1 重新索引 Reindexing\r\n",
    "\r\n",
    "类方法：`.reindex(index, method=, fill_value=)`\r\n",
    "\r\n",
    "参数：\r\n",
    "\r\n",
    "+ `index=`：用于重新索引的新数组；\r\n",
    "+ `method=`：插值填充的方式；\r\n",
    "+ `fill_value=`：缺失位置填充的值；\r\n",
    "+ `limit=`：向前、先后填充时的最大填充量；\r\n",
    "+ `level=`：在 `MultiIndex` 的指定级别上匹配简单索引，否则选取子集；\r\n",
    "+ `copy=`：默认为 `True`，无论何时都复制，如果设置为 `Fales`，则新旧相等时不复制；\r\n",
    "\r\n",
    "插值处理：\r\n",
    "\r\n",
    "+ `method=ffill` 或 `method=pad`：向前（forward）填充数据\r\n",
    "+ `method=bfill` 或 `method=backfill`：向后（backward）填充数据"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "source": [
    "# Series的重新索引\r\n",
    "\r\n",
    "ser1_1 = pandas.Series([4.5, 7.2, -5.3, 3.6], index=[\"d\", \"b\", \"a\", \"c\"])\r\n",
    "\r\n",
    "ser1_2 = ser1_1.reindex([\"a\", \"b\", \"c\", \"d\", \"e\"])                  # 不存在的值将被设为NaN\r\n",
    "ser1_3 = ser1_1.reindex([\"a\", \"b\", \"c\", \"d\", \"e\"], fill_value=0)    # 对不存在的值统一填充为0\r\n",
    "\r\n",
    "arr_info([ ser1_1, ser1_2, ser1_3 ])"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "source": [
    "# Series的插值处理\r\n",
    "\r\n",
    "ser1_4 = pandas.Series([\"Red\", \"Green\", \"Blue\"] ,index=[0, 2, 4])\r\n",
    "ser1_5 = ser1_4.reindex(range(6), method=\"pad\")\r\n",
    "\r\n",
    "arr_info([ ser1_4, ser1_5 ])"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "source": [
    "# DataFrame的重新索引\r\n",
    "\r\n",
    "frame1_1 = pandas.DataFrame(numpy.arange(12).reshape(4,3), index=[\"color_1\", \"color_3\", \"color_4\", \"color_5\"], columns=[\"R\",\"G\",\"B\"])\r\n",
    "\r\n",
    "frame1_2 = frame1_1.reindex(index=[\"color_1\", \"color_2\", \"color_3\", \"color_4\", \"color_5\"])    # 对行进行reindex\r\n",
    "frame1_3 = frame1_1.reindex(columns=[\"R\", \"G\", \"B\", \"Hex\"])     # 对列进行reindex（由于第一个位置参数是index，所以必须用关键词参数）\r\n",
    "\r\n",
    "# 也可以同时对行列进行reindex\r\n",
    "frame1_4 = frame1_1.reindex(index=[\"color_1\", \"color_2\", \"color_3\", \"color_4\", \"color_5\"], columns=[\"R\", \"G\", \"B\", \"Hex\"])\r\n",
    "# 系统Warning：使用关键词参数代替位置参数以避免歧义（ambiguity），将来如果使用位置参数将报错TypeError\r\n",
    "\r\n",
    "arr_info([ frame1_1, frame1_2, frame1_3, frame1_4 ])"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "source": [
    "# DataFrame的插值处理\r\n",
    "# 注：教材指出插值只能按行（axis=0），但实际上可能新版本更改了，也可以按列插值\r\n",
    "\r\n",
    "frame1_5 = frame1_1.reindex(index=[\"color_1\", \"color_6\", \"color_3\", \"color_4\", \"color_5\"], method=\"ffill\")  # 对行插值\r\n",
    "frame1_6 = frame1_1.reindex(columns=[\"R\", \"G\", \"B\", \"Hex\"], method=\"ffill\")     # 对列插值\r\n",
    "\r\n",
    "frame1_7 = frame1_1.reindex(index=[\"color_1\", \"color_6\", \"color_3\", \"color_4\", \"color_5\"], \r\n",
    "                            columns=[\"R\", \"G\", \"B\", \"Hex\"], method=\"ffill\") # 对行列同时插值\r\n",
    "# 好像是按照排序后的顺序，来确定前后被复制的是谁\r\n",
    "\r\n",
    "arr_info([ frame1_5, frame1_6, frame1_7 ])"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "source": [
    "# ix方法\r\n",
    "# frame1_1.ix[[\"color_1\", \"color_2\", \"color_3\", \"color_4\", \"color_5\"], [\"R\", \"G\", \"B\", \"Hex\"]] 报错，猜测是新版本删去了此方法"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "### 5.2.2 丢弃指定轴上的项 Dropping entries from an axis\r\n",
    "\r\n",
    "类方法：`pandas_obj.drop(index, axis=0)`"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "source": [
    "# Series\r\n",
    "\r\n",
    "ser2_1 = pandas.Series(range(5), index=[\"a\", \"b\", \"c\", \"d\", \"e\"])\r\n",
    "\r\n",
    "ser2_2 = ser2_1.drop(\"c\")           # 丢弃单行\r\n",
    "ser2_3 = ser2_1.drop([\"d\", \"c\"])    # 丢弃多行（需要传入列表）\r\n",
    "\r\n",
    "arr_info([ ser2_1, ser2_2, ser2_3 ])"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "source": [
    "# DataFrame\r\n",
    "\r\n",
    "frame2_1 = pandas.DataFrame(numpy.arange(12).reshape(4,3), index=[\"a\", \"b\", \"c\", \"d\"], columns=[\"R\", \"G\", \"B\"])\r\n",
    "\r\n",
    "# 丢弃行\r\n",
    "frame2_2 = frame2_1.drop(\"b\")           # 默认按行，这里实际上省略了关键词参数：axis=0\r\n",
    "frame2_3 = frame2_1.drop([\"a\", \"c\"])    # 默认按行，这里实际上省略了关键词参数：axis=0\r\n",
    "\r\n",
    "# 丢弃列\r\n",
    "frame2_4 = frame2_1.drop(\"R\", axis=1)\r\n",
    "frame2_5 = frame2_1.drop([\"G\", \"R\"], axis=1)\r\n",
    "\r\n",
    "arr_info([ frame2_1, frame2_2, frame2_3, frame2_4, frame2_5 ])"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "### 5.2.3 索引，选取，以及过滤 Indexing, selection, and filtering"
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "#### 5.2.3.1 Series的索引\r\n",
    "\r\n",
    "Series的索引：`ser_object[index]`\r\n",
    "\r\n",
    "+ 索引：类似NumPy，但除了可以使用整数索引（区间:`[s,t)`），还能使用标签进行索引（包含结束元素，区间:`[s,t]`）；\r\n",
    "+ 切片：整数切片、标签切片\r\n",
    "+ 花式索引：整数、标签；\r\n",
    "+ 布尔索引"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "source": [
    "# Series的索引\r\n",
    "\r\n",
    "ser3_1 = pandas.Series(range(5), index=[\"a\", \"b\", \"c\", \"d\", \"e\"])\r\n",
    "arr_info([ ser3_1 ])"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "source": [
    "## 标签索引、整数索引、花式索引、布尔索引\r\n",
    "arr_info([ ser3_1[1] ])                     # 整数索引\r\n",
    "arr_info([ ser3_1[\"b\"] ])                   # 标签索引\r\n",
    "arr_info([ ser3_1[2:4] ])                   # 整数切片，区间：[s,t)\r\n",
    "arr_info([ ser3_1[\"c\":\"e\"] ])               # 标签切片，区间：[s,t]。【注意】这里与整数切片不同，包含结束元素！\r\n",
    "arr_info([ ser3_1[ [1, 3] ] ])              # 整数索引的花式索引\r\n",
    "arr_info([ ser3_1[ [\"b\", \"a\", \"d\" ] ] ])    # 标签索引的花式索引\r\n",
    "arr_info([ ser3_1[ ser3_1 < 2 ] ])          # 布尔索引"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "source": [
    "## 切片的赋值\r\n",
    "ser3_1[\"c\":\"e\"] = 5\r\n",
    "arr_info([ ser3_1 ])"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "#### 5.2.3.2 DataFrame的索引\r\n",
    "\r\n",
    "DataFrame的（直接）索引：`dataframe[\"lab_c\"]`\r\n",
    "\r\n",
    "+ 索引：直接索引时，只能对列标签进行索引 \r\n",
    "+ 切片：整数切片、标签切片\r\n",
    "+ 花式索引：直接索引时，仅能对列标签进行花式索引；\r\n",
    "+ 布尔索引\r\n",
    "\r\n",
    "【注】这里“直接索引时”指的是不调用以下类方法进行索引。自创词汇，非标准名称；\r\n",
    "\r\n",
    "**(NEW)** DataFrame的索引选项：\r\n",
    "\r\n",
    "选取行：\r\n",
    "+ `df.loc[\"lab_r\"]` 用标签索引选取行 Selects single row or subset of rows from the DataFrame by label\r\n",
    "+ `df.iloc[i]`：用整数索引选取行 Selects single row or subset of rows from the DataFrame by integer position\r\n",
    "+ `df.xs(\"lab_r\")`\r\n",
    "\r\n",
    "选取列：\r\n",
    "+ `df[\"lab_c\"]`：用标签索引选取列 Select single column or sequence of columns from the DataFrame; special case conveniences: boolean array (filter rows), slice (slice rows), or boolean DataFrame (set values based on some criterion)\r\n",
    "+ `df.loc[:, \"lab_c\"]`：用整数索引选取列 Selects single column or subset of columns by label\r\n",
    "+ `df.iloc[:, j]`：选择第 j 列 Selects single column or subset of columns by integer position\r\n",
    "\r\n",
    "选取元素\r\n",
    "+ `df.loc[\"lab_r\", \"lab_c\"]` Select both rows and columns by label\r\n",
    "+ `df.iloc[i, j]` Select both rows and columns by integer position\r\n",
    "+ `df.at[\"lab_r\", \"lab_c\"]`：用标签索引选取指定行列元素 Select a single scalar value by row and column label\r\n",
    "+ `df.iat[i, j]`：用整数索引选取指定行列元素 Select a single scalar value by row and column position (integers)\r\n",
    "\r\n",
    "其他：\r\n",
    "+ `reindex`：method Select either rows or columns by labels\r\n",
    "+ `get_value`, `set_value`：methods Select single value by row and column label\r\n",
    "\r\n",
    "【注意】\r\n",
    "+ 如果是数字标签，则既有可能是整数类型的，也有可能是字符类型的，取决于定义时的类型；\r\n",
    "+ `\"lab_r\"`、`\"lab_c\"` 分别表示“行标签”、“列标签”，但需要注意的是，如果标签的类型是整数，则不应该加引号；\r\n",
    "+ `.ix[]` 属性已在新版本中去除，请用 `.loc[]`、`.iloc[]` 代替；"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "source": [
    "# DataFrame的索引\r\n",
    "\r\n",
    "frame3_1 = pandas.DataFrame(  { \"Nation\": [\"BR\", \"UK\", \"CN\", \"GR\"], \r\n",
    "                                \"City\": [\"Rio\", \"London\", \"Beijing\", \"Athens\"], \r\n",
    "                                \"Year\": [2016, 2012, 2008, 2004] \r\n",
    "                                }, \r\n",
    "                            index=[\"31th\", \"30th\", \"29th\", \"28th\"]  )\r\n",
    "arr_info([ frame3_1 ])\r\n",
    "arr_info([ frame3_1[\"City\"] ])  # 直接对DataFrame索引只能对列进行，以下均报错：df[\"29th\"]、df[2]、df[2][3]"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "source": [
    "## 选取行\r\n",
    "arr_info([ frame3_1.loc[\"29th\"] ])\r\n",
    "arr_info([ frame3_1.iloc[2] ])\r\n",
    "arr_info([ frame3_1.xs(\"29th\") ])       # 选取某行：根据标签选取某行（列标签会报错），返回Series\r\n",
    "arr_info([ frame3_1[2:3] ])             # 选取某行：frame[2] 报错，可以用切片代替取当行"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "source": [
    "## 选取列\r\n",
    "arr_info([ frame3_1[\"City\"] ])          # DataFrame的直接索引是取出一列，返回Series\r\n",
    "arr_info([ frame3_1.loc[:, \"City\"] ])\r\n",
    "arr_info([ frame3_1.iloc[:, 2] ])\r\n",
    "arr_info([ frame3_1[\"City\":\"Year\"] ])   # 标签切片是左闭右闭区间，因此这样选取为空列"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "source": [
    "# DataFrame的切片\r\n",
    "\r\n",
    "## 直接切片\r\n",
    "arr_info([ frame3_1[1:4] ])             # 整数切片\r\n",
    "arr_info([ frame3_1[\"30th\":\"28th\"] ])   # 标签切片。注意：只能用index进行标签切片（例如：[\"UK\":\"CN\"]是无效的）"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "source": [
    "## 利用类方法切片\r\n",
    "\r\n",
    "### 对行进行切片\r\n",
    "arr_info([ frame3_1.loc[\"31th\":\"29th\"] ])   # 标签切片是左闭右闭区间\r\n",
    "arr_info([ frame3_1.iloc[1:4] ])\r\n",
    "\r\n",
    "### 对列进行切片\r\n",
    "arr_info([ frame3_1.loc[:, \"Nation\":\"City\"] ])  # 标签切片是左闭右闭区间\r\n",
    "arr_info([ frame3_1.iloc[:, 0:2] ])"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "source": [
    "# DataFrame的花式索引\r\n",
    "\r\n",
    "arr_info([ frame3_1[ [\"City\", \"Year\"] ] ])  # 直接花式索引（只能对列进行）\r\n",
    "arr_info([ frame3_1.loc[ \"29th\", [\"City\", \"Year\"] ] ])"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "source": [
    "# DataFrame的布尔索引\r\n",
    "\r\n",
    "frame3_2 = pandas.DataFrame(numpy.arange(12).reshape(4,3), index=[\"[1]\", \"[2]\", \"[3]\", \"[4]\"], columns=[\"A\", \"B\", \"C\"])\r\n",
    "arr_info([ frame3_2 ])\r\n",
    "\r\n",
    "arr_info([ frame3_2[frame3_2 <= 3] ])\r\n",
    "\r\n",
    "frame3_2[frame3_2 > 6] = 1      # 使用布尔条件赋值\r\n",
    "arr_info([ frame3_2 ])"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "### 5.2.4 整数索引 Integer Indexes\r\n",
    "\r\n",
    "Working with pandas objects indexed by integers is something that often *trips up* (绊倒) new users due to some differences with indexing semantics on built-in Python data structures like lists and tuples.\r\n",
    "\r\n",
    "对于非整数索引，`series[\"lab_r\"]`、`series[index_r]` 不会引起歧义，但对于整数索引，如果数据对象本身的索引是整数类型的，那么可能带来潜在的问题。\r\n",
    "\r\n",
    "例如，`series[1]` 是指标签为 1，还是索引为 1 ？或者说，是指 `series.loc[1]` 还是 `series.iloc[1]`？\r\n",
    "\r\n",
    "因此，这种情况请尽量运用上节提到的 `.loc[]`、`.iloc[]` 代替"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "source": [
    "# 更加严谨地使用索引\r\n",
    "\r\n",
    "ser4_1 = pandas.Series(numpy.arange(9), index=list(\"abcdefghi\"))\r\n",
    "arr_info([ ser4_1 ])\r\n",
    "\r\n",
    "arr_info([ ser4_1[-1] ])    # 如果index为整数，这样引用后新版本已经报错（更加严格）\r\n",
    "arr_info([ ser4_1.loc[\"f\"] ])\r\n",
    "arr_info([ ser4_1.iloc[3] ])"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "### 5.2.5 算数运算与数据对齐 Arithmetic and data alignment\r\n",
    "\r\n",
    "进行算术操作时，如果索引相同，则对应元素执行计算，如果索引不重叠（overlap），将以 `NaN` 代替。DataFrame的将同时在行和列上进行数据对齐。\r\n",
    "\r\n",
    "Flexible arithmetic methods:\r\n",
    "+ `add`, `radd` Methods for addition (+)\r\n",
    "+ `sub`, `rsub` Methods for subtraction (-)\r\n",
    "+ `div`, `rdiv` Methods for division (/)\r\n",
    "+ `floordiv`, `rfloordiv` Methods for floor division (//)\r\n",
    "+ `mul`, `rmul` Methods for multiplication (*)\r\n",
    "+ `pow`, `rpow` Methods for exponentiation (**)"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "source": [
    "# Series的数据对齐\r\n",
    "\r\n",
    "ser4_1 = pandas.Series([7.3, -2.5, 3.4, 1.5], index=[\"a\", \"c\", \"d\", \"e\"])\r\n",
    "ser4_2 = pandas.Series([-2.1, 3.6, -1.5, 4, 3.1], index=[\"a\", \"c\", \"e\", \"f\", \"g\"])\r\n",
    "\r\n",
    "arr_info([ser4_1, ser4_2])\r\n",
    "arr_info([ser4_1 + ser4_2])"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "source": [
    "# DataFrame的数据对齐\r\n",
    "\r\n",
    "frame4_1 = pandas.DataFrame(numpy.arange(12).reshape(3,4), index=list(\"bcd\"), columns=[\"C\", \"C++\", \"Java\", \"Python\"])\r\n",
    "frame4_2 = pandas.DataFrame(numpy.arange(12).reshape(4,3), index=list(\"dbae\"), columns=[\"C\", \"Python\", \"HTML\"])\r\n",
    "arr_info([frame4_1, frame4_2])\r\n",
    "\r\n",
    "arr_info([frame4_1 + frame4_2])"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "#### 5.2.5.1 在算数方法中填充值 Arithmetic methods with fill values\r\n",
    "\r\n",
    "+ `fill_value=`：当轴标签在一个数据对象（Series或DataFrame）中存在，而在另一个数据对象中未找到时，将会填充指定的值；\r\n",
    "+ 如果两个数据对象中都为 `NaN`，将不会填充指定的值；\r\n",
    "\r\n",
    "类方法：\r\n",
    "+ `.add()`\r\n",
    "+ `.sub()`\r\n",
    "+ `.div()`\r\n",
    "+ `.mul()`"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "source": [
    "# 算术运算\r\n",
    "\r\n",
    "addition = frame4_1.add(frame4_2, fill_value=0)\r\n",
    "subtraction = frame4_1.sub(frame4_2, fill_value=0)\r\n",
    "divide = frame4_1.div(frame4_2, fill_value=0)       # 除以0的结果将以inf（无穷）表示\r\n",
    "multiply = frame4_1.mul(frame4_2, fill_value=0)\r\n",
    "\r\n",
    "ser4_1.add(ser4_2, fill_value=0)    # Series与DataFrame的用法相同，不再赘述\r\n",
    "\r\n",
    "arr_info([ addition, subtraction, divide, multiply ])"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "source": [
    "# 重新索引并填充数据\r\n",
    "\r\n",
    "# 分别取二者行、列标签的并集，并重新索引（reindex）\r\n",
    "row_label = [\"a\", \"b\", \"c\", \"d\", \"e\"]\r\n",
    "column_label = [\"C\", \"C++\", \"HTML\", \"Java\", \"Python\"]\r\n",
    "\r\n",
    "ri_frame4_1 = frame4_1.reindex(index=row_label, columns=column_label)\r\n",
    "ri_frame4_2 = frame4_2.reindex(index=row_label, columns=column_label)\r\n",
    "\r\n",
    "arr_info([ ri_frame4_1, ri_frame4_2 ])\r\n",
    "# 通过对比可以看到，以下5个元素在两个DataFrame中都不存在（NaN）：\r\n",
    "# (a, C++)  (e, C++)  (c, HTML)  (a, Java)  (e, Java)"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "#### 5.2.5.2 DataFrame与Series之间的操作 Operations between DataFrame and Series\r\n",
    "\r\n",
    "+ 默认情况下，DataFrame与Series之间计算时，Series的 `index` 将匹配到DataFrame的列标签（`columns`），然后按行向下进行广播。即DataFrame的每行分别与该Series进行计算。\r\n",
    "+ 如果 `index` 在Series中或DataFrame中均找不到，将会以所有 `index` 的并集进行重索引，并以 `NaN` 代替不存在的值；\r\n",
    "+ 默认匹配DataFrame的列标签，如果要匹配DataFrame的行标签，需要用相应的类方法，例如 `.add(series, axis=0)` 来指定轴参数。The axis number that you pass is the *axis* to match on.；\r\n",
    "+ 注意：`index=[0,1,2]` 与 `index=list(\"012\")` 所得到的索引不能相互识别并合并"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "source": [
    "# NumPy中不同维度数组之间的计算\r\n",
    "\r\n",
    "arr4_1 = numpy.arange(12).reshape(3, 4)\r\n",
    "arr4_2 = arr4_1[0]\r\n",
    "\r\n",
    "arr_info([arr4_1, arr4_2])\r\n",
    "arr_info([arr4_1 - arr4_2])     # 不同维度数组之间的计算称之为广播（broadcast），将在12章介绍"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "source": [
    "# DataFrame与Series之间的计算\r\n",
    "\r\n",
    "frame4_3 = pandas.DataFrame(numpy.arange(12).reshape(3,4), index=[1,2,3], columns=list(\"ABCD\"))\r\n",
    "ser4_3 = frame4_3.xs(1)     # 取一行\r\n",
    "# ser4_4 = frame4_3[\"A\"]      # 取一列\r\n",
    "\r\n",
    "arr_info([frame4_3, ser4_3])\r\n",
    "arr_info([frame4_3 + ser4_3])"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "source": [
    "# 当index在Series中或DataFrame中均找不时\r\n",
    "\r\n",
    "ser4_4 = pandas.Series([0,1,2,3], index=list(\"ACDE\"))\r\n",
    "\r\n",
    "arr_info([frame4_3, ser4_4])\r\n",
    "arr_info([frame4_3 + ser4_4])"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "source": [
    "# 按列进行广播\r\n",
    "\r\n",
    "ser4_5 = frame4_3[\"B\"]\r\n",
    "ser4_6 = pandas.Series([1,3,5], index=list(\"ABC\"))\r\n",
    "\r\n",
    "arr_info([ frame4_3, ser4_5, ser4_6 ])\r\n",
    "arr_info([ frame4_3.add(ser4_5, axis=0) ])\r\n",
    "arr_info([ frame4_3.add(ser4_6) ])          # 实际上默认是按行进行广播（即axis参数为1）\r\n",
    "arr_info([ frame4_3.add(ser4_6, axis=0) ])  # 如果设定Series的index与列不匹配的话，会按index的并集重索引"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "source": [
    "# 注意index的数据类型，index=[\"1\",\"2\",\"3\"]和index=[1,2,3]是不相同的\r\n",
    "\r\n",
    "ser4_7 = pandas.Series([7, 5, 2, 0, 1], index=[0, \"1\", 1, 2, 3])\r\n",
    "\r\n",
    "arr_info([ frame4_3.add(ser4_7, axis=0) ])\r\n",
    "print(\"注意index的数据类型:\", ser4_7.index) # 注意index 1 和 \"1\" 并没有被合并"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "### 5.2.6 函数应用与映射 Function application and mapping\r\n",
    "\r\n",
    "+ `object.apply(data, axis=)`\r\n",
    "+ `series.map(data, axis=)`\r\n",
    "+ `dataframe.applymap(data, axis=)`"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "source": [
    "# NumPy的通用函数（ufunc）也适用于pandas的数据对象\r\n",
    "\r\n",
    "ser5_1 = pandas.Series([3,1,5,2,7,8,6,4,9], index=list(\"abcdefghj\"))\r\n",
    "frame5_1 = pandas.DataFrame(numpy.arange(1, 13).reshape(3, 4), index=[\"一\", \"二\", \"三\"], columns=list(\"ABCD\"))\r\n",
    "arr_info([ ser5_1, frame5_1 ])\r\n",
    "\r\n",
    "## 数学函数\r\n",
    "arr_info([ numpy.sqrt(ser5_1) ])\r\n",
    "arr_info([ numpy.sqrt(frame5_1) ])\r\n",
    "\r\n",
    "## 统计函数\r\n",
    "arr_info([ numpy.std(ser5_1, ddof=1) ])\r\n",
    "arr_info([ numpy.std(frame5_1, ddof=1) ])"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "source": [
    "# 将函数应用于各行或各列：.apply()方法\r\n",
    "\r\n",
    "function = lambda x: x.max() - x.min()   # lambda关键词：匿名函数\r\n",
    "\r\n",
    "arr_info([ frame5_1.apply(function, axis=0) ])  # 计算每列。axis默认为0，可省略\r\n",
    "arr_info([ frame5_1.apply(function, axis=1) ])  # 计算每行\r\n",
    "\r\n",
    "arr_info([ frame5_1.sum() ])    # 很多功能已有相应的类方法，可以直接调用，例如：.sum()、.std()、.mean()"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "source": [
    "# .apply()可以返回多个值，例如返回Series\r\n",
    "def func(x):\r\n",
    "    return pandas.Series([x.min(), x.max()], index=[\"MIN\", \"MAX\"])\r\n",
    "\r\n",
    "\r\n",
    "arr_info([ frame5_1.apply(func, axis=0) ])  # 计算每列。axis默认为0，可省略\r\n",
    "arr_info([ frame5_1.apply(func, axis=1) ])  # 计算每行"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "source": [
    "# 对Series进项逐元素（Element-wise）操作：.map()类方法\r\n",
    "\r\n",
    "format_data = lambda x: \"%.2f\" %x\r\n",
    "\r\n",
    "arr_info([ ser5_1.map(format_data) ])"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "source": [
    "# 对DataFrame进项逐元素（Element-wise）操作：.applymap()类方法\r\n",
    "\r\n",
    "format_data = lambda x: \"%.2f\" %x\r\n",
    "\r\n",
    "arr_info([ frame5_1.applymap(format_data) ])"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "### 5.2.7 Sorting and Ranking 排序与排名"
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "#### 5.2.7.1 排序（Sorting）\r\n",
    "\r\n",
    "对索引进行排序：\r\n",
    "\r\n",
    "+ `series.sort_index(ascending=True)`\r\n",
    "+ `dataframe.sort_index(axis=, ascending=True)`\r\n",
    "\r\n",
    "对值进行排序：\r\n",
    "\r\n",
    "+ `series.sort_values(ascending=True)`\r\n",
    "+ `dataframe.sort_values(axis=, ascending=True)`\r\n",
    "+ 【弃用】`.order()`\r\n",
    "\r\n",
    "缺失数据时默认将 `NaN` 放在最后\r\n",
    "\r\n",
    "多个轴都排序，优先排序第一个标签所在列，重复值处以第二个标签所在列排序"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "source": [
    "# Series的排序\r\n",
    "\r\n",
    "ser6_1 = pandas.Series([2,4,5,1,3], index=list(\"deacb\"))\r\n",
    "arr_info([ ser6_1 ])\r\n",
    "\r\n",
    "## 按索引进行排序\r\n",
    "arr_info([ ser6_1.sort_index() ])\r\n",
    "arr_info([ ser6_1.sort_index(ascending=False) ])    # 降序"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "source": [
    "## 按数值进行排序\r\n",
    "arr_info([ ser6_1.sort_values() ])\r\n",
    "arr_info([ ser6_1.sort_values(ascending=False) ])   # 降序"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "source": [
    "## 缺失数据时的情况\r\n",
    "ser6_2 = pandas.Series([2,4,numpy.NaN,1,3], index=list(\"deacb\"))\r\n",
    "arr_info([ ser6_2.sort_values() ])  # 默认将NaN放在最后"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "source": [
    "# DataFrame的排序\r\n",
    "\r\n",
    "frame6_1 = pandas.DataFrame({\"B\": [5,1,2,4,3], \"A\": [9,6,9,8,7]}, index=list(\"deacb\"))\r\n",
    "arr_info([ frame6_1 ])\r\n",
    "\r\n",
    "## 按索引进行排序\r\n",
    "arr_info([ frame6_1.sort_index() ])         # 默认以axis=0排序，可省略\r\n",
    "arr_info([ frame6_1.sort_index(axis=1) ])   # 以axis=1排序"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "source": [
    "## 按数值进行排序\r\n",
    "arr_info([ frame6_1.sort_values(by=\"A\") ])          # by参数不能省略\r\n",
    "arr_info([ frame6_1.sort_values(by=[\"A\", \"B\"]) ])   # 多个轴都排序，优先排序第一个标签所在列，重复值处以第二个标签所在列排序"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "#### 5.2.7.2 排名（Ranking）\r\n",
    "\r\n",
    "+ `series.rank(ascending=, method=)`\r\n",
    "+ `dataframe.rank(axis=, ascending=, method=)`\r\n",
    "\r\n",
    "method选项：\r\n",
    "\r\n",
    "+ `method=average`：\r\n",
    "+ `method=min`：\r\n",
    "+ `method=max`：\r\n",
    "+ `method=first`："
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "source": [
    "# Series\r\n",
    "\r\n",
    "ser6_3 = pandas.Series([7, -5, 7, 4, 2, 0, 4])\r\n",
    "\r\n",
    "arr_info([ ser6_3.rank() ])\r\n",
    "arr_info([ ser6_3.rank(method=\"first\") ])\r\n",
    "arr_info([ ser6_3.rank(method=\"average\") ])"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "source": [
    "# DataFrame\r\n",
    "\r\n",
    "frame6_2 = pandas.DataFrame({\"B\": [4.3, 7, -3, 2], \"A\": [0, 1, 0, 1], \"C\": [-2, 5, 8, -2.5] })\r\n",
    "\r\n",
    "arr_info([ frame6_2 ])\r\n",
    "arr_info([ frame6_2.rank() ])       # 默认按axis=0进行\r\n",
    "arr_info([ frame6_2.rank(axis=1) ])"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "### 5.2.8 带有重复值的轴索引 Axis indexes with duplicate values\r\n",
    "\r\n",
    "+ `obj.index.is_unique`：查看index的值是否唯一。注意这是一个属性，不是类方法；\r\n",
    "+ 如果Series或DataFrame的索引对应多个值，将分别返回一个Series或DataFrame\r\n",
    "+ 如果该索引对应单个值，将返回一个元素"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "source": [
    "# Series\r\n",
    "\r\n",
    "ser7_1 = pandas.Series(range(5), index=[\"a\", \"a\", \"b\", \"b\", \"c\"])\r\n",
    "\r\n",
    "arr_info([ ser7_1 ])\r\n",
    "print(\"\\n判断index的值是否唯一:\", ser7_1.index.is_unique, \"\\n\")\r\n",
    "arr_info([ ser7_1[\"a\"] ])   # 如果该索引对应多个值，将返回一个Series\r\n",
    "arr_info([ ser7_1[\"c\"] ])   # 如果该索引对应单个值，将返回一个元素"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "source": [
    "# DataFrame\r\n",
    "\r\n",
    "frame7_1 = pandas.DataFrame(numpy.arange(12).reshape(3, 4), index=[0, 0, 1], columns=[\"A\", \"A\", \"B\", \"B\"])\r\n",
    "\r\n",
    "arr_info([ frame7_1 ])\r\n",
    "arr_info([ frame7_1[\"A\"] ])     # 取所有 列 含columns=\"A\"的列\r\n",
    "arr_info([ frame7_1.xs(0) ])    # 取所有 行 含index=0行"
   ],
   "outputs": [],
   "metadata": {}
  }
 ],
 "metadata": {
  "orig_nbformat": 4,
  "language_info": {
   "name": "python",
   "version": "3.9.6",
   "mimetype": "text/x-python",
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "pygments_lexer": "ipython3",
   "nbconvert_exporter": "python",
   "file_extension": ".py"
  },
  "kernelspec": {
   "name": "python3",
   "display_name": "Python 3.9.6 64-bit ('venv': venv)"
  },
  "interpreter": {
   "hash": "2df30c634058628fc5df5036be3dee25b811a252316c0aa1ff7f50eb8aecb5be"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}